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ABSTRACT 

Host standardised testing prograas do not furnish 
examples of the kinds of inforaation young people actually know. 
Instead they indicate only (1) how far an individual student is above 
or below an average score, and (2) the average score of a classrooa 
or a school in relation to others. In contrast, National Assessaent 
evaluates educational changes within the larger population, its 
purpose being to provide aore adequate inforaation about educational 
quality on a regional and a national basis. Based on a model of 
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describes the knowledge, skills, attitudes, and understandings of 
groups of persons at the ages Of ?, 13, 17, and young adults. 

National Assessment measures "population knowledge," rather than the 
knowledge of an individual. Results for science and citizenship show 
two ttendst (1) knowledge of students Increases with age, and (2) 
adults tend to fotget auch of what they learned In cchool. 
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NATIONAL ASSESSMENT - - EVALUATION AND ACCOUNTABILITY ‘ 

' - ■ j ! ; By -j ... •• 

I / • / ( George B. Brain ; * 

From the puberty rights o £ the upper Paleolithic period to present-day forms 

* * ' ' ' - • * . A i I ‘ 4 * *' / * ' - , ’ 

of objective tests, evaluation and assessment have played important roles in 
educational planning. Man has always felt the need of defining certain charac- 

, • j , , ^ 

(eristics that were needed for successful functioning. Coniorming behavior 
in a social setting became the first major criterion for this kind of assessment. 

. r ‘ • r ' , • c ' - . ■ ■ ■■ ’ . ' * 

Certain present or past characteristics were deemed to be predictive of 
future congruence to the criterion. 1 - ’ . 

4 Though the early predictors were variously assessed in terms of manlinoss, 
hunting prowess/ knowledge of sexual mores, and recognition of authority they 
emerged in early socitles as meaningful dimensions of evaluation or assessment. 

The more specific criteria of effective human conduct were basically re- 
lated to the stable predictable patterns of behavior which were deemed consonant 
with the survival of the group. Eventually, certain kinds of behavior developed 
into powerful means of education. So early man in his own trial and error s 
method found'that certain characteristics that were observable oi measurable 
were reliable predictors of stable group behavior. 

» x ; •<. • -jV ^'v: 'C : ' ry ■' /.L; 

In order to insure the criteria of effective and predictable human behavior, 

: y _ pr v\y- , y •? ’ » y. . ?y \..y, 

a number of mechanisms of control emerged. These mechanisms had as their 

) . yy; . , ■ j • - y ./•«;. ; • • . y . ; ; 

'goal the collective security of the group. They are now Identifiable as the ; 

. , j - ,- F ■ > .... "■ .tr ' " • • >. ■ -i ^ r - -x . - ’ . ' 

: .. v V...- ' .. • ? \ .X f •• • • •>' f -• • 

Umi\y f cudtomi/ rellgloui belled, and education. To early 

man, they represented a concrete framework by which the cultural heritage 
was transmitted and the survival of lha group insured. * ; 
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The earliest form of assessment was global in nature. Though no attempt 
was mode to separate intellectual functioning from personality! the behavioral 
patterns of the individual were subjected to the criterion of group collective 
security. • ■ ' »■;./'/ .• ' _ 

i , 

- * i • 

The early leaders of men were empirical psychologists. Homer and the 
Bible provide examples of the psychological wisdom of early men. 

Ulysses on one occasion, not wishing to go to war, pretended to bo mad. 

He began to plow a field sowing salt. One of his friends placed his young son 

. ^ 4 . * . ' ” ^ " 5 - " ••• V ' ' ' 

directly in front of the plow; and Ulysses turned aside, showing Indeed that he 
was not mad and could not escape fulfilling his promises. 

The Bible, too, is filled with descriptions of how behavior was assessed. 

In the account of Genesis, Adam hid himself and God is portrayed as being 
aware that something was wrong because of this change in Adam's behavior. 

Likewise, in one of the early tribal wars, Gilead found a way to distinguish 
between friendly and unfriendly groups. He asked them to pronounce the word 
•’Shibboleth. " If they could not pronounce it correctly, they were judged the 



• '■ * * ' : ,. v t ‘ . -V. 

enemy. 4 •. y v'V: 

Thus the oral examination plus prescribed feats of courage served as a 
predictor of effective behavior among early men. Later on as formal education 



developed, the degree of memory recall and conformity to school expectations 
also became a part of the evaluation procedure, * . ? ; 

:• -r i . . - v, r.. . -V*'.. /i ^ " •&}. I ■: v • ' V ! • . .. " ,;'V . ■' ••••. ' ■ ■> ;■ 1: ^ . 



But It was not long before a theoretical framework was joined to the 
observation process. Plato in Ms hierarchy of society demarcated certain 
kinds of behavior expected by various levels of society. He expected the 



philosopher rulers to manifest verbal ability and intellectual wisdom, the soldier 
to evidence courage and manliness, and the vast massos to be concerned only 
with bodily functions. - v . V- ■ 

Hippocrates discerned differences in men's constitutional make-up suggest- 

' ; i t .... r ■ ‘ . • 

ing that all individuals fell into the categories of phlegmatic, choleric, sanguine, 
or melancholic. Vw'':' : 

Within the nineteenth century, further developments took place in the 

* ’ I * 

assessment of behavior. An extension of the oral examination was developed 
in the essay examination. During the present century, methods of measurement 
used in the physical and biological sciences have been gradually introduced 

% • ' i \ - : ; •• 

■ ' ’ i": « ■ 

Into the study of human behavior. This approach has been sparked by a desire 
to become more objective in the analysis of human behavior. 

Accompanying the expansion of testing programs, a new technological „ 
development, the use of scoring machines and computers, has made the analysis 
of data easy and economical to obtain. 

The fantastic developments In computer technology and data processing 
illustrate dramatically the new dimensions of assessment procedures. It is 
possible to compare Individuals or groups of individuals with large samples and 
norms developed from these large samples. 

But though the procedures have become exceedingly advanced It is quite 
apparent that evaluation of pupil behavior moved away from the original Intent 
of assessment procedures. \y i ■ 

The early procedures were related to making judgments about the predicted 
success or failure of an individual within a specific cultural setting. 
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The recent direction of evaluation, despite its technical and methodological 
excellence, subtly introduced a now criterion: comparison with the average. 

• . ...V: •• '■■■l'-..'/.:- ■ » „ ^ 

Explicity or implicitly, the test score became the criterion in itsolf rather 
than a means to an end. The mathematical symbol became all too often tho 
criterion of expected performance without any real reference to the bohavioral 
phenomenon needed for success. The purpose of the National Assessment 
design is to return to the idea of making judgments about groups of individuals 
within specific cultural settings, ' 

Man's earliest sociological groups held that :ertain stable patterns of 
behavior were necessary to maintain the security of the group, engage in 
collectl^ food gathering, and provide for mutual protection. The extended 
family, customs, law, ethics, religion, priesthood, chieftains, police functions, 
and education all seem to have developed as man's unique response to his 
environment. These controls, however, were chiefly imposed on man from 
without. The individual had little choice but to conform to these external con- 
irol mechanisms. 4 Another bolstering factor in this control was the fact that 
the family wai often anchored to the land and dependent on an agrarian economy. 

Today the characteristics of life in America have changed radically. A 
I960 survey showed that one out of five Americans changed his place of residence 
within that year. As a result the typical control mechanisms of the pact have 

. * • - .v ••••.•;. ■ . .. ■' y ■" * '' . 

•" •• J-. "l; '• .. ^ • V - b'[ 

weakened, giving rise to the charge by many that there is a sense of cultural 
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drift in American values. 

Mobility today means that in moat instances the family er.tsts alone and 



independently in a strange area. Very seldom do uncles, aunts, or cousins, 
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grandparents, and others reside next door or evun in the same geographical 
region. Mobility means that the significance of property as a psychological 
anchor and root has also been changed, Banks indicate that the average'longth 
of occupancy of a house with a 25-year mortgage is 7 years. 

Changes have resulted in sex roles with women coming to marriage not 
with feather beds, cooking ability, and canning prowoss but with professional 
competencies, such as teaching, nursing, or secretarial work. 

Mobility has resulted in changing patterns of housing, such as are seen 
in the vast suburban developments where relatively homogeneous groupings 
are imposed by qualifying salary criteria and contracting agreements. 

Changes in tfie strength of institutional religion have also occurred. It is 
a well-known fact that a large part of our population changes its religious 
affiliation because of marriage or personal preference. 

Methods of child rearing have alac changed with accent on more permissive 
patterns. From more direct controls, the family has learned to rely on more 
indirect ones. Boys get less punishment and girls less indulgence and pro- 
tectiveness. 'Though girls are possibly less debilitated by parental affection 
and power, boys find it more difficult to identify clear roles to envy or emulate. 
The end result may be that the procedures themselves are less effective in the 



development Kt leadership and responsibility. ; 

To all of these changes must be added the continual crisis orientation of 



modern politital developments with the need for competitive achievement In 



the areas of space satellites, moon probes, and atomic arsenals for defense. 
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The international aceno for the past 20 years has beep one of continued peaks of 
anxiety about global destruction and warfare. 

Federal programs often In response to International demands or conditions 
have affected the huge segments of our population through educational programs 
and defense contract spending. Meanwhile riots and demonstrations have mark- 
ed the difficult struggle of American citizens to deal with the adaptations required. 
The cultural anchors of man's psychological security have been uprooted and 
man's challenge to the alternative of being controlled f/om external sources 

• : •• • y • ... • . j • t 

has produced the necessity of developing controls from within. Essentially, 
man's social and psychological orientation has not kept pace with the changes 
th&t are occurring all too rapidly in the dimensions of technological change. 

Thus in my view, assessment once again has emerged as a sort of cultural 

control mechanism, but there has been added to It the Idea of public accountability. 

V ‘ r - • ■ > ■ • . v- ■ . ’ 

Assessment Investigators change over a period of time for the various sub- 
ject areas included In the assessment program. And that in and of itself can 
prove to be Its most significant contribution. ‘ , 

Since public education la in the political domain, many decisions affecting 
education are made on the basis of the self-interest of the decision makers. A 
vote In the Ceneral Assembly: a judgment by the Governor; decisions by super- 
intendents! teachers, and principals; votes. In school board meetings; and reso- 
lutlons and actions of educational organisations often tend to reflect Interests 
other than the prime constituency of learning -- students in schools. Where 
decisions In public education do not reflect some genuine attempts to move 
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students more substantially toward goals of educational quality, then the publics 
served by the schools increasingly will demand public accountability for those 

decisions. •'Y‘ V • - 

Obviously there are difficulties in measuring relationships between educa- 
tional decisions and quality in education. However, the trend is toward 
development of educational goals, assessment of student performance and 
evaluation of effective and qualitative ichool services and processes. This 
sequence is becoming more concrete all the time, and it will be used to demand 
accountability from those who make decisions in public education. 

So now the schools must deal with evaluation, assessment, and accountability. 

Any accountability program should seek to develop a consensus on educa- 
tional goals and objectives. It should identify the school services and processes 
which can advance students toward objectives and goals as well as those which 
impede student achievement. From this sequence of goals -objectives -services 
and processes, schools should develop a system which can be used at all 
levels of educational decision making to appraise those decisions within the 
framework of goals and means to those goals. The central question should be 
•to what oxlent are decisions affecting school services and processes advancing 
or impeding student growth and changes in behavior with respect to specific 
objectives and broader goals?*' 

Increasingly the American public is becoming output oriented. Cltitens 







realise that the success of many things can be measured by specific outcomes-* 
such as the landing of an Apollo flight on the moon-> or business success through 




investment growth or profits. Obviously we in education are a long way from 



v’;.- H-'ti'-xM&i % Offers B- : ; i 






knowing what wo should know about relationships among school services, pro- 
cesses, achievement and goals, not to mention the impact of out-of- school student 
life and environment on learning. But the lime has come to make quality in 
education a truly operational force for improving the quality of mankind. Evalua- 
tion, Assessment or Accountability must not be viewed with alarm or concern by 
school boards or school officials, but as tools to be employed with professional 
precision for improving the quality and the output of tho public educational 
enterprise for which they are legally responsible. Wo»are entering an era when 
every institution, public or private, will be held to a new level of accountability 



by fedoral and state agencies of government working at new levels of sophisti- 
cation. V . * 



■ - . i • *- v . » 
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Moat school standardised testing programs in use today do not furnish 
examples of the kinds of information young people actually know, Instead they 
indicate: (1) how far a particular student is above or below an average score; 
and (2) how far the average score of classroom or school is above or below 
others with which it Is compared. ’ ^ ^ ; ^ 

National Assessment is designed to describe the knowledge, skills, attitudes 
and understandings of persons in four selected age groups. The results con- 




sequently sample what these young oeople know and do not know. Assessment ■ 

employs statistical sampling procedures to determine what certain groups of 

^ cm » n * ; + v vv ; ' -c ■■ ■■ ■ v: ; o 



people know, rather than what certain individuals know. National Assessment 
Is a measure of "population knowledge" not the knowledge of an individual. 

A standardised test Is a different device. The usual test, standardised or 
otherwise, is designed to yield a score for each individual who takes the test. 



That score indicatcF where a student ranks in relation to othor students, how 
he rates in relation to a norm or standard. . % 

The customary standardized teat is constructed to moasure the ability or 
achievement of an individual, and also can be used to compare classes or 
schools. ■■ : i; .’ 1 

Standardized tosts are extremely useful and essential devices for teachers, 
administrators and other educators. National Assessment is not a substitute 
for these education functions. On the other hand, National Assessment is 
an evaluation of educational changes within the larger population, and its purpose 
is to provide more adequate information on the quality and success oi educational 
efforts on a regional and national basis. » * 

To know that Johnny scores a. the 90th percentile on a standardized reading 
test is perhaps Interesting to Johnny and to Johnny's parents, teacher and 
school; but It tells nothing about the specific kind of material that Johnny can 
actually raad or ‘ r stand. Standardized tests do not provide a descriptive 
analysis of the ««. tuai kinds of knowledge or skills young people, as a whole, 
have or do no\ have. r .. ;/«■ r •>*>.>. .... 

National Assessment was specifically designed not to provide a score for 

• v i /f, \ ■ >- ' ' " v 

each individual. On the contrary, assessment was created on the model of 

stetistlcal sampling such as is used in pabyc*oplnlon polls, to establish a 
census of actual knowledge of groups of persons in the United States et the ages 
of 9, 1 i, 19, and young adults, Not every individual must participate In the 
assessment, because the sample Is chosen on statistically sound grounds, U 
Is possible to generalise from the response of the individuals to the larger group 
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National Assessment differs from standardised tests in other ways. 

National Assessment uses a greater variety of exercise formats. For example, 
there were various kind j of equipment used in the exercises on Science.. The 
students used these devices to demonstrate their knowledge and skill in U3ing 
scientific equipment. The assessment in Writing actually involved writing, ; . ' 



not just asking questions about writing practices and techniques. The Citizen- 
ship assessment included evaluation of students in group discussions. : 



Another difference is that while standardized tests are generally admin- 
istered so that a student reads each question to himself, the National Assessment 
model provides for a taped voice to read all instructions and exercises while 
the student reads along in the booklet in front of him. This method was used 
with groups of about 12 students who were assembled to take the assessment v ; 
package. A few packages were administered to individual students, and an 
interviewer then read each exercise aloud and wrote down the student'6 response. 



V 



For the adult assessment, each adult was interviewed in his home by a trained 



interviewer. 



% V. 



#' Finally, 'the method of administration of assessment is different from the 
usual test. National Assessment hired its own field staff to contact schools, draw 
the sample of students or adults, and administer the packages of exercises. No 

’i ^ r s r 4 - , * > * . . * . . i t A i 

school personnel were involved in the administration of the exercises to students, 



,v 



The method 



other than to assist with basic arrangements for the assessment. 

of hiring a permanent field Ptaff, plus using taped reading of instructions and 

' ' ■" ’ ' ■ ‘^\ ' * ' ‘V»'2 ‘"-'.si . v ’ ■ ' Ivi-.f': <■ V : : '• ■ - ■’ 

; f exercises, increased the comparability of the assessment procedure used all , 

, ft ' <: 1 $? \ ' .V. • '"ft £,*" - ■ ft'.*- ‘ ■••H * • • ’ *• ’* ■ : 

f ' over the country, as well as lessening any burden a local school might feel if • * ' 

ERJC ft- j - . . ft -ft ; '’ft;. : . ' • ft ft "ft fftftftv ; ft/, ft, • • ; 'ft - ft/ft ’ • ■ / ft- [ ■ ft 1 o 
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askad to conduct the administration of exercises itself. Moreover, it added to 
the credibility of the process. 

There are a number of immediate benefits from the National Assessment 
program. National Assessment has provided: 1 ’ / 

1. v The first national description of samples of learning behavior among 



j , selected populations of the nation. These tell how as a nation we 

&*'. stack up educationally. ' vV '.'."r:' 






2. f Clarification of the difference between the traditional testing for 
sorting purposes and evaluating the attainment levels of selected 



. 1t$- "■ ■ : ' 

■ V 

r : E, 



population groups. National Assessment provides a new model for ' 

'• .-i : V-' . r‘, f ' • ' 'a 'V 

V; assessing education attainment. ; 



3. Demonstration that educational objectives of a school can be formulated 



m; v 



M i,\ 



and agreed upon in a way that involves parents, laymen and school 
staffers with a resultant increase in common^under standing of what 
objectives the schools are attempting to accomplish. National 






o accomplish. National 
• Assessment has provided one workable model for citizen involvement 
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in sdtting educational objectives, 



Demonstration that assessment exercises can be constructed to pro- 



; i : ,,- V vide information about the entire range of school children, not merely 

>v- : : ^v ' ■ ■»— ' ’ v/- • '•?=■ < ■ .<-■ - ‘-v • ^ 
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: . ?. data at out the average child. National Assessment indicates what the 



;• V 



lowest and highest ability ranges know. \ 

fMi ■- 5. Demonstration of the feasibility and desirability of using a variety of i . ^ 

' , '••Xvi' techniques (e. g. questionnaires, interviews, and performance, etc.) ■ 

I rather than just traditional paper and pencil\tests. In the past it has 
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been believed that every pupil must be given every appraisal exercise 
and hence too much time would be required if several techniques were 
V/r‘; used, but National Assessment employs a method which divides 
l - ■'//'/■. exercises into booklets, or packages so that no one pupil takes > 

, m ore than a fraction of the total; yet, achievements of a total pupil i ' 

group can be measured* The techniques of assessment administration 
(i.e, written material plus tape recording) were designed to minimize 

a. ^3 « ■ m ^ ^ •• • v * . . * ** ? » ' k*- 



V:- ' reading limitations. . . . . . 

• * v l • r - - v 1,;^- ’-.v' '» V. ' k> v*^ v j'u v v *. k ' //..•{ : Vv.‘ ' v 

6. In addition to these direct benefits of National Assessment, a number 



. of "by product" benefits can be utilized by local school leaders. For 
Vy ; example*, since National Assessment provides specific information on 
• ■ the strengths and weaknesses in educational achievement, the program 



design should contribute to more effective curriculum development and 

;-v *■ •• -.Cy i,, 

assist local school officials in developing programs of emphasis within 
their school system. ■ ’ - . ; , , • 
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National Assessment will provide a substantial source of data useful y 

I 



for further research into and understanding of the problems of the 
central cities, suburbs, differing geographic areas, and communities, 



. X 



as well as Black and non- Black, etc. : National Assessment is a new 

*/ ’ "yV;' ^ >••*&> y :-i AV 

and important collection of information about the United States which 



will have many diverse applications, y; 
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Two major trends --one comforting and the other disturbing- -emerge from 



analysis of the answers for science and citizenship, the only two areas reported 

L ERJCi ■ >-/ • ’ V / 



1.‘ The knowledge of students increases with age. K 

E. Adults tend to forget much of what they learned in school. 

In the vast majority of exercises in both subject areas, the percentage of 



respondents answering correctly rose from age 9 thru 13 to 17. And on other 
questions, mainly technical ones in science, fewer adults than 17-year-olds 



gave the correct answer, -^yv. > \-|»j .'.V, - V : ‘ v ‘ [,:x . - , 

For instance, more 13-year-olds than 9-year-olds picked 70 degrees as 
the most comfortable temperature for a school room, and more 17-year-olds 



than 13-year-olds knew that the Senate is oneof the two houses of Congress. 



• But in the science area, adults did not do as well as 17-year-olds in such • . 






things as identifying Charles Darwin as the author of the th-'ory of evolution by 



natural selection. 
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The major exceptions were on questions that involved practical knowledge 



rather than book learning. A.dults outscored students whenever knowledge had 



to be applied in a practical context 
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There were items that. most respondents knew, like where babies come from 
and the name of the President. * ,H‘ v ‘ 

; Few trends were identifiable but the most striking were the tendency of 









children with educated parents to do best, and the fact that black students per- 
formed as well or nearly as well as whites-on the science questions. > 



Many of the exercises on citizenship were designed to gauge attitudes and 
opinions rather than knowledge. , ' ' 

. 1 Answers to one series of questions showed that adults tend to be apathetic 
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* in civic action. For instance, 86 per cent cited ways in which citizens can 



5y5jj233325SEB«l2!I53 * " « - . T m - ■ 'f - • , ^ „ • ? * * ^ ^ : * .** • : .• - ; ■. .* ^ _ .y « ^ ... ^ ^ 



influence government, but only 61 per cent thought they could influence their own 
state officials. And 61 per cent could think of reasons to write pub ic officials, 
but only 27 per cent said they had done so. 



More 17-year-olds than adults felt they should act to stop an instance of racial 
discrimination, and more 13 and 17-year-olds said they would be willing to have 






someone of another race as a neighbor or doctor. 
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National Assessment will, it is hoped, raise a great many questions among 
both educators and interested citizens. Aside from suth a general and vital 



question as "Where do we stand, where are we going, where should we plan to 



go?" A direct benefit of the Assessment is the posing of a number of questions 



for exploration by the education community. Among them: 



v S'VH .. A Si' 



1. Should state and local school systems make their own assessments? 



y'-' 
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2. Will the states find it desirable, practical or politic to develop 



programs for interstate comparisons? 

’ r’.-' *■' l , ; '. sri'T 1 :- fi' 



3. 



Should test publishers make assessment-like instruments available 
to states and local schools? . .'■'■■•"•d; •, 






4. Can the National Assessment model, using its sampling techniques, 



experiences in objectives and exercise formulation, be adapted to . 



.; W V ' 

% : 



smaller geographical units such as a school district? : \ 



; 5. Do some subjects, such as reading and mathematics, lend themselves 

; ' : i to "national objectives" without posing a threat to local control of V. ~ 



. K 'I-" ■ 

■ curricula? 



6. If National Assessment data prove meaningful should Assessment 






f activities (the evaluation of other subject areas) be speeded up (at 



considerable increase in costs)? 
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